Search for: All records

Existing preference optimization objectives for language model alignment require additional hyperparameters that must be extensively tuned to achieve optimal performance, increasing both the complexity and time required for fine-tuning large language models. In this paper, we propose a simple yet effective hyperparameter-free preference optimization algorithm for alignment. We observe that promising performance can be achieved simply by optimizing inverse perplexity, which is calculated as the inverse of the exponentiated average log-likelihood of the chosen and rejected responses in the preference dataset. The resulting simple learning objective, SimPER, is easy to implement and eliminates the need for expensive hyperparameter tuning and a reference model, making it both computationally and memory efficient. Extensive experiments on widely used real-world benchmarks, including MT-Bench, AlpacaEval 2, and 10 key benchmarks of the Open LLM Leaderboard with 5 base models, demonstrate that SimPER consistently and significantly outperforms existing approaches—even without any hyperparameters or a reference model. For example, despite its simplicity, SimPER outperforms state-of-the-art methods by up to 5.7 points on AlpacaEval 2 and achieves the highest average ranking across 10 benchmarks on the Open LLM Leaderboard. The source code for SimPER is publicly available at: https://github.com/tengxiao1/SimPER. 

This work studies the alignment of large language models with preference data from an imitation learning perspective. We establish a close theoretical connection between reinforcement learning from human feedback (RLHF) and imitation learning (IL), revealing that RLHF implicitly performs imitation learning on the preference data distribution. Building on this connection, we propose DIL, a principled framework that directly optimizes the imitation learning objective. DIL provides a unified imitation learning perspective on alignment, encompassing existing alignment algorithms as special cases while naturally introducing new variants. By bridging IL and RLHF, DIL offers new insights into alignment with RLHF. Extensive experiments demonstrate that DIL outperforms existing methods on various challenging benchmarks. The code for DIL is available at https://github.com/tengxiao1/DIL. 

We report a narrow-linewidth laser based on thin-film lithium tantalate (TFLT). The laser is composed of an InP reflective semiconductor optical amplifier gain chip hybrid integrated with a TFLT waveguide external cavity cladded with a silicon oxide extended Bragg grating. The single-frequency laser device achieves an on-chip output power of approximately 26 mW and an intrinsic Lorentzian linewidth of ~94 Hz. These results highlight the great potential of TFLT for integrated photonic laser applications, enabling high-coherence and high-power laser sources in a compact platform. 

We study the problem of aligning large language models (LLMs) with human preference data. Contrastive preference optimization has shown promising results in aligning LLMs with available preference data by optimizing the implicit reward associated with the policy. However, the contrastive objective focuses mainly on the relative values of implicit rewards associated with two responses while ignoring their actual values, resulting in suboptimal alignment with human preferences. To address this limitation, we propose calibrated direct preference optimization (Cal-DPO), a simple yet effective algorithm. We show that substantial improvement in alignment with the given preferences can be achieved simply by calibrating the implicit reward to ensure that the learned implicit rewards are comparable in scale to the ground-truth rewards. We demonstrate the theoretical advantages of Cal-DPO over existing approaches. The results of our experiments on a variety of standard benchmarks show that Cal-DPO remarkably improves off-the-shelf methods. 

Abstract The invention of the laser unleashed the potential of optical metrology, leading to numerous advancements in modern science and technology. This reliance on lasers, however, also introduces a bottleneck for precision optical metrology, as it requires sophisticated photonic infrastructure for precise laser-wave control, leading to limited metrology performance and significant system complexity. Here, we take a key step toward overcoming this challenge by demonstrating a Pockels laser with multifunctional capabilities that elevate optical metrology to a new level. The chip-scale laser achieves a narrow intrinsic linewidth down to 167 Hz and a broad mode-hop-free tuning range up to 24 GHz. In particular, it delivers an unprecedented frequency chirping rate of up to 20 EHz/s and an exceptional modulation bandwidth exceeding 10 GHz, both of which are orders of magnitude greater than those of existing lasers. Leveraging this laser, we successfully achieve velocimetry at 40 m/s over a short distance of 0.4 m, and measurable velocities up to the first cosmic velocity at 1 m away—a feat unattainable with conventional ranging approaches. At the same time, we achieve distance metrology with a ranging resolution of <2 cm. Furthermore, for the first time to our knowledge, we implement a dramatically simplified architecture for laser frequency stabilization by directly locking the laser to an external reference gas cell without requiring additional external light control. This approach enables long-term laser stability with a frequency fluctuation of only ±6.5 MHz over 60 min. The demonstrated Pockels laser combines elegantly high laser coherence with ultrafast frequency reconfigurability and superior multifunctional capability. We envision its profound impact across diverse fields including communication, sensing, autonomous driving, quantum information processing, and beyond. 

Abstract Optical microcomb underpins a wide range of applications from communication, metrology, to sensing. Although extensively explored in recent years, challenges remain in key aspects of microcomb such as complex soliton initialization, low power efficiency, and limited comb reconfigurability. Here we present an on-chip microcomb laser to address these key challenges. Realized with integration between III and V gain chip and a thin-film lithium niobate (TFLN) photonic integrated circuit (PIC), the laser directly emits mode-locked microcomb on demand with robust turnkey operation inherently built in, with individual comb linewidth down to 600 Hz, whole-comb frequency tuning rate exceeding 2.4 × 10¹⁷ Hz/s, and 100% utilization of optical power fully contributing to comb generation. The demonstrated approach unifies architecture and operation simplicity, electro-optic reconfigurability, high-speed tunability, and multifunctional capability enabled by TFLN PIC, opening up a great avenue towards on-demand generation of mode-locked microcomb that is of great potential for broad applications. 

Abstract Soliton microcombs are a promising new approach for photonic-based microwave signal synthesis. To date, however, the tuning rate has been limited in microcombs. Here, we demonstrate the first microwave-rate soliton microcomb whose repetition rate can be tuned at a high speed. By integrating an electro-optic modulation element into a lithium niobate comb microresonator, a modulation bandwidth up to 75 MHz and a continuous frequency modulation rate up to 5.0 × 10¹⁴Hz/s are achieved, several orders-of-magnitude faster than existing microcomb technology. The device offers a significant bandwidth of up to tens of gigahertz for locking the repetition rate to an external microwave reference, enabling both direct injection locking and feedback locking to the comb resonator itself without involving external modulation. These features are especially useful for disciplining an optical voltage-controlled oscillator to a long-term reference and the demonstrated fast repetition rate control is expected to have a profound impact on all applications of frequency combs. 

Abstract Understanding dynamic human mobility changes and spatial interaction patterns at different geographic scales is crucial for assessing the impacts of non-pharmaceutical interventions (such as stay-at-home orders) during the COVID-19 pandemic. In this data descriptor, we introduce a regularly-updated multiscale dynamic human mobility flow dataset across the United States, with data starting from March 1st, 2020. By analysing millions of anonymous mobile phone users’ visits to various places provided by SafeGraph, the daily and weekly dynamic origin-to-destination (O-D) population flows are computed, aggregated, and inferred at three geographic scales: census tract, county, and state. There is high correlation between our mobility flow dataset and openly available data sources, which shows the reliability of the produced data. Such a high spatiotemporal resolution human mobility flow dataset at different geographic scales over time may help monitor epidemic spreading dynamics, inform public health policy, and deepen our understanding of human behaviour changes under the unprecedented public health crisis. This up-to-date O-D flow open data can support many other social sensing and transportation applications. 

High-fidelity periodic poling over long lengths is required for robust, quasi-phase-matched second-harmonic generation using the fundamental, quasi-TE polarized waveguide modes in a thin-film lithium niobate (TFLN) waveguide. Here, a shallow-etched ridge waveguide is fabricated in x-cut magnesium oxide doped TFLN and is poled accurately over 5 mm. The high fidelity of the poling is demonstrated over long lengths using a non-destructive technique of confocal scanning second-harmonic microscopy. We report a second-harmonic conversion efficiency of up to 939 %.W⁻¹(length-normalized conversion efficiency 3757 %.W⁻¹.cm⁻²), measured at telecommunications wavelengths. The device demonstrates a narrow spectral linewidth (1 nm) and can be tuned precisely with a tuning characteristic of 0.1 nm/°C, over at least 40 °C without measurable loss of efficiency.